Skip to content

restore option to compress raster#944

Open
melonora wants to merge 14 commits into
scverse:mainfrom
melonora:restore_storage_options
Open

restore option to compress raster#944
melonora wants to merge 14 commits into
scverse:mainfrom
melonora:restore_storage_options

Conversation

@melonora
Copy link
Copy Markdown
Collaborator

@melonora melonora commented Jun 10, 2025

closes #37

This PR adds the capability back of using either lz4 or zstd compression at various compression levels. It is exposed through a compressor argument in write and write_element that takes as value a dict with key being the compression name and the value being the compression level. By default if compression is specified bytes will be reordered for more efficient compression SHUFFLE in blosc.

@melonora
Copy link
Copy Markdown
Collaborator Author

perhaps still some refactoring to clean up the code a bit, but first want opinions on the exposed parameter.

@melonora melonora mentioned this pull request Jun 10, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 10, 2025

Codecov Report

❌ Patch coverage is 76.47059% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.96%. Comparing base (d8bf265) to head (d3607f1).

Files with missing lines Patch % Lines
src/spatialdata/_io/io_raster.py 71.79% 11 Missing ⚠️
src/spatialdata/_io/_utils.py 90.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #944      +/-   ##
==========================================
- Coverage   92.06%   91.96%   -0.11%     
==========================================
  Files          51       51              
  Lines        7792     7841      +49     
==========================================
+ Hits         7174     7211      +37     
- Misses        618      630      +12     
Files with missing lines Coverage Δ
src/spatialdata/_core/spatialdata.py 91.96% <100.00%> (+<0.01%) ⬆️
src/spatialdata/_io/_utils.py 86.88% <90.00%> (+0.13%) ⬆️
src/spatialdata/_io/io_raster.py 89.30% <71.79%> (-3.92%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@LucaMarconato
Copy link
Copy Markdown
Member

LucaMarconato commented Jan 4, 2026

Thanks @melonora, I picked this up and merged the conflicts arising after the Zarr v3 support.

The tests currently fail mainly due to the disconnect between the new format options and dask, but it should be straightforward to make the PR green again after https://github.com/ome/ome-zarr-py/pull/510/changes is merged.

@LucaMarconato
Copy link
Copy Markdown
Member

Sharding support has been added to ome-zarr-py ome/ome-zarr-py#534 and is being added to spatialdata via #1106.

LucaMarconato and others added 3 commits May 12, 2026 17:39
Use "compressors" (plural) with native zarr v3 codec objects (ZstdCodec,
BloscCodec) in storage_options, which ome-zarr-py >= 0.16.0 + dask >= 2026.3.0
correctly forwards to zarr_array_kwargs. Bump minimum deps accordingly and fix
test assertions for lz4 (.clevel) vs zstd (.level) and the s0 array path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@LucaMarconato
Copy link
Copy Markdown
Member

@melonora I merged the latest main and bumped the min ome-zarr and dask versions to be aligned with the sharding PR. Now the tests pass.

Can you please do a review of the changes? If you also think it's ok, feel free to merge (before the sharding PR).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Re-enable compression

2 participants